Methods and systems for representing usage of an electronic learning system

ABSTRACT

A method and system for representing usage of an electronic learning system. The method and system involve receiving an input indicative of a selection of a property of the electronic learning system to be represented, the property including a course content provided by the electronic learning system and an activity available for that course content; receiving representation parameters that define a scope of the usage to be represented; determining an event count for the property during the period of interest; and generating a usage indicator for the property based at least on the event count. The usage indicator is generally reflective of at least a usage amount of the property during the period of interest.

CROSS-REFERENCE TO RELATED PATENT APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/425,048, filed on May 29, 2019, which is a continuation of U.S.patent application Ser. No. 14/267,023 filed on May 1, 2014 and issuedas U.S. Pat. No. 10,339,616 on Jul. 2, 2019. The contents of each of theforegoing are hereby incorporated by reference.

TECHNICAL FIELD

The described embodiments relate to methods and systems associated withrepresenting usage of an electronic learning system, and in particular,to representing usage of interaction with a course content provided byan electronic learning system.

INTRODUCTION

Electronic learning (also known as “e-Learning” or “eLearning”)generally refers to education or learning where users engage ineducation related activities using computers and other computingdevices. For example, users may enroll or participate in a course orprogram of study offered by an educational institution (e.g., a college,university or grade school) through a web interface that is accessibleover the Internet. Users may receive assignments electronically,participate in group work and projects by collaborating over theInternet, and be graded based on assignments and examinations that aresubmitted, for example, using an electronic submission tool.

Electronic learning is not limited to use by educational institutions.Electronic learning may be used in other environments, such asgovernment and corporations. For example, employees at a regional branchoffice of a corporation may use electronic learning to participate in atraining course offered by another office, or even a third-partyprovider. As a result, the employees at the regional branch office canparticipate in the training course without having to travel to the siteproviding the training course. Travel time and costs can be reduced andconserved.

In addition to offering convenient access to electronic learning,electronic learning systems can collect and manage data associated withthe usage of the systems. In traditional learning environments,instructors are typically unable to properly assess the value of certaincourse content. Although course assessments can provide the instructorswith an estimate of the value of the course contents, the instructorscannot readily determine how or whether the course contents were used.Therefore, the instructors are usually unable to correlate a student'sresult in the course with the provided course contents.

SUMMARY OF SOME EMBODIMENTS

The various embodiments described herein generally relate to methods(and associated systems configured to implement the methods) forrepresenting usage of an electronic learning system.

In accordance with one embodiment, there is provided a method forrepresenting usage of an electronic learning system. The method includesreceiving an input indicative of a selection of an interaction with acourse content provided by the electronic learning system; receivingrepresentation parameters that define a scope of the usage to berepresented, the representation parameters including at least a periodof interest; determining an event count for the interaction during theperiod of interest, the event count being a number of events stored atone or more storage components of the electronic learning system for theinteraction with the course content; and generating a usage indicatorthat is reflective of a usage amount of the course content during theperiod of interest, the usage indicator being generated by adjusting theevent count with at least one of a count parameter and a weight factor,the count parameter providing at least one limitation to a value of theevent count and the weight factor indicating an amount of influence thatthe interaction has on the usage of the electronic learning system.

In some embodiments, the representation parameters further comprise atime interval that is shorter than the period of interest; anddetermining the event count for the interaction during the period ofinterest includes separating the period of interest into a set ofsub-periods according to the time interval, a length of at least onesub-period corresponds to the time interval; and determining a pluralityof counts for the period of interest, each count being the number ofevents corresponding to the interaction with the course content during asub-period and the event count being a sum of the plurality of counts.

In some embodiments, the count parameter includes at least one of anupper count limit and a lower count limit; and adjusting the event countwith the count parameter includes, when the count parameter is the uppercount limit: determining whether the event count exceeds the upper countlimit; and in response to determining the event count exceeds the uppercount limit, reducing the event count to the upper count limit; and,when the count parameter is the lower count limit: generating anintermediary event count by subtracting the lower count limit from theevent count; and if the intermediary event count is less than zero,setting the event count to zero, otherwise using the intermediary eventcount as the event count.

In accordance with another embodiment, there is provided a method forrepresenting usage of an electronic learning system. The methodcomprising: receiving an input indicative of a selection of a propertyof the electronic learning system to be represented, the propertyincluding a course content provided by the electronic learning systemand an activity available for that course content; receivingrepresentation parameters that define a scope of the usage to berepresented, the representation parameters including at least a periodof interest; determining an event count for the property during theperiod of interest, the event count corresponding to a number of eventsstored at one or more storage components of the electronic learningsystem in association with the property for the period of interest; andgenerating a usage indicator for the property based at least on theevent count, the usage indicator being reflective of at least a usageamount of the property during the period of interest.

In some embodiments, the representation parameters include a timeinterval, the time interval being shorter than the period of interest;and determining the event count for the property during the period ofinterest comprises: separating the period of interest into a set ofsub-periods according to the time interval, a length of at least onesub-period corresponds to the time interval; and determining a pluralityof counts for the period of interest, each count being the number ofevents corresponding to the property for a sub-period and the eventcount being a sum of the plurality of counts.

In some cases, generating the usage indicator for the property based atleast on the event count comprises: determining a top count from the twoor more counts, the top count being the count from the two or morecounts with a greatest value; and normalizing each count with the topcount.

In some cases, the method further includes receiving a transformationmode indicator for the event count, the transformation mode indicatorproviding a type of adjustment to be applied to the event count; andvarying each count in accordance with the transformation mode indicator.

In some cases, the transformation mode indicator corresponds to a peakmode; and varying each count in accordance with the transformation modeindicator comprises: identifying the sub-period associated with the topcount; and, for each sub-period subsequent to the identified sub-period,replacing the corresponding normalized count with the normalized topcount.

In some cases, the transformation mode indicator corresponds to acumulative average mode; and varying each count in accordance with thetransformation mode indicator comprises, for each sub-period, retrievingthe count associated with each prior sub-period; and generating anadjusted count by determining an average of the count for each priorsub-period and the count for that sub-period; and replacing the countfor that sub-period with the adjusted count.

In some cases, determining the event count for the property during theperiod of interest further comprises: setting the event count to zero;retrieving, from the one or more storage components, events that satisfythe representation parameters; and for each retrieved event, determiningwhether that event is associated with the property and if so,incrementing the event count.

In some cases, generating the usage indicator for the property based atleast on the event count comprises: receiving a weight factor for theactivity, the weight factor indicating an amount of influence theactivity has on the usage of the electronic learning system; andadjusting the event count with the weight factor.

In some cases, determining the event count for the property during theperiod of interest further comprises: receiving a count parameter forthe event count, the count parameter providing at least one limitationto a value of the event count; and adjusting the event count based onthe count parameter.

In some cases, the count parameter comprises an upper count limit;adjusting the event count based on the count parameter comprises:determining whether the event count exceeds the upper count limit; andin response to determining the event count exceeds the upper countlimit, reducing the event count to the upper count limit.

In some cases, the count parameter comprises a lower count limit;adjusting the event count based on the count parameter comprises:generating an intermediary event count by subtracting the lower countlimit from the event count; and if the intermediary event count is lessthan zero, setting the event count to zero, otherwise using theintermediary event count as the event count.

In some cases, the activity is associated with a first course contentand a second course content that is different from the first coursecontent; the usage indicator comprises a first usage indicator for theactivity at the first course content and a second usage indicator forthe activity at the second course content; determining the event countfor the property during the period of interest comprises: generating afirst event count for the first course content, the first event countcorresponding to the usage amount of the activity at the first coursecontent during the period of interest; and generating a second eventcount for the second course content, the second event countcorresponding to the usage amount of the activity at the second coursecontent during the period of interest; and generating each of the firstusage indicator and the second usage indicator based at least on thefirst event count and the second event count.

In some cases, the method may further include determining whether thefirst event count is greater than the second event count; and if thefirst event count is greater than the second event count, normalizingeach of the first event count and the second event count with the firstevent count, otherwise normalizing each of the first event count and thesecond event count with the second event count.

In some cases, generating the usage indicator for the property comprisesproviding a visual representation of the usage indicator. The visualrepresentation may include a heat map. The visual representation mayinclude an adjustable display operable to receive one or more controlinputs for selecting a designated time within the period of interest;and vary the visual presentation to illustrate the usage indicator atthe designated time.

In some cases, the property further comprises a group composed of one ormore users of the electronic learning system.

In some cases, the activity comprises at least an interaction by a userwith the electronic learning system.

In some cases, the course content comprises any one or more componentsthat provides course material, a discussion forum, a chat, anassessment, a rubric, an assignment, a message board, a checklist, and asubmission tool.

In accordance with another embodiment, there is provided a system forrepresenting usage of an electronic learning system. The systemcomprising: a processing component configured to operate one or morecomponents of the electronic learning system to represent usage of theelectronic learning system; one or more storage components for storing,at least, events associated with the electronic learning system, the oneor more storage components being in communication with the processingcomponent; and a representation component operated by the processingcomponent to: receive an input indicative of a selection of a userinteraction type with a course content provided by the electroniclearning system; receive representation parameters that define a scopeof the usage to be represented, the representation parameters includingat least a period of interest; determine an event count for the userinteraction type during the period of interest, the event count being anumber of events stored at the one or more storage components of theelectronic learning system for the user interaction type with the coursecontent; and generate a usage indicator that is reflective of a usageamount of the course content during the period of interest, the usageindicator being generated by adjusting the event count with at least oneof a count parameter and a weight factor, the count parameter providingat least one limitation to a value of the event count and the weightfactor indicating an amount of influence that the user interaction typehas on the usage of the electronic learning system.

In some cases, the representation parameters further comprise a timeinterval that is shorter than the period of interest; and therepresentation component is further operated by the processing componentto: separate the period of interest into a set of sub-periods accordingto the time interval, a length of at least one sub-period corresponds tothe time interval; and determine a plurality of counts for the period ofinterest, each count being the number of events corresponding to theuser interaction type with the course content during a sub-period andthe event count being a sum of the plurality of counts.

In some cases, the count parameter comprises at least one of an uppercount limit and a lower count limit; and the representation component isfurther operated by the processing component to: when the countparameter is the upper count limit: determine whether the event countexceeds the upper count limit; and in response to determining the eventcount exceeds the upper count limit, reduce the event count to the uppercount limit; and when the count parameter is the lower count limit:generate an intermediary event count by subtracting the lower countlimit from the event count; and if the intermediary event count is lessthan zero, set the event count to zero, otherwise using the intermediaryevent count as the event count.

In accordance with another embodiment, there is provided a system forrepresenting usage of an electronic learning system. The systemcomprising: a processing component configured to operate one or morecomponents of the electronic learning system to represent usage of theelectronic learning system; one or more storage components for storing,at least, events associated with the electronic learning system, the oneor more storage components being in communication with the processingcomponent; and a representation component operated by the processingcomponent to: receive an input indicative of a selection of a propertyof the electronic learning system to be represented, the propertyincluding a course content provided by the electronic learning systemand an activity available for that course content; receiverepresentation parameters that define a scope of the usage to berepresented, the representation parameters including at least a periodof interest; determine an event count for the property during the periodof interest, the event count corresponding to a number of events storedat the one or more storage components of the electronic learning systemin association with the property for the period of interest; andgenerate a usage indicator for the property based at least on the eventcount, the usage indicator being reflective of at least a usage amountof the property during the period of interest.

DETAILED DESCRIPTION OF THE DRAWINGS

Several embodiments will now be described in detail with reference tothe drawings, in which:

FIG. 1 is a schematic diagram of components interacting with anelectronic learning system in accordance with some embodiments;

FIG. 2 is a block diagram of some components that may be implemented inthe electronic learning system in accordance with an example embodiment;

FIG. 3 is a flowchart diagram of an example method for representingusage of the electronic learning system;

FIGS. 4A and 4B are screenshots of example user interfaces for receivingvarious inputs by the electronic learning system;

FIG. 5A is a table showing event counts for a property of the electroniclearning system in accordance with an example embodiment;

FIG. 5B is a table showing adjusted event counts generated based on theevent counts in FIG. 5A in accordance with an example embodiment;

FIG. 5C is a table showing usage indicator values generated based on theadjusted event counts in FIG. 5B in accordance with an exampleembodiment;

FIG. 5D is a screenshot of an example graphical representation of theusage indicator values in FIG. 50 ;

FIG. 6A illustrates two tables showing event counts for two coursecontents provided by the electronic learning system in accordance withan example embodiment;

FIG. 6B illustrates two tables showing adjusted event counts generatedbased on the event counts in FIG. 6A in accordance with an exampleembodiment; and

FIG. 6C is a screenshot of a graphical representation generated based onthe adjusted event counts in FIG. 6B in accordance with an exampleembodiment.

The drawings, described below, are provided for purposes ofillustration, and not of limitation, of the aspects and features ofvarious examples of embodiments described herein.

DESCRIPTION OF SOME EMBODIMENTS

For simplicity and clarity of illustration, elements shown in thedrawings have not necessarily been drawn to scale. The dimensions ofsome of the elements may be exaggerated relative to other elements forclarity. It will be appreciated that for simplicity and clarity ofillustration, where considered appropriate, reference numerals may berepeated among the drawings to indicate corresponding or analogouselements or steps. In addition, numerous specific details are set forthin order to provide a thorough understanding of the exemplaryembodiments described herein. However, it will be understood by those ofordinary skill in the art that the embodiments described herein may bepracticed without these specific details. In other instances, well-knownmethods, procedures and components have not been described in detail soas not to obscure the embodiments generally described herein.Furthermore, this description is not to be considered as limiting thescope of the embodiments described herein in any way, but rather asmerely describing the implementation of various embodiments asdescribed.

The embodiments of the systems and methods described herein may beimplemented in hardware or software, or a combination of both. In somecases, embodiments may be implemented in one or more computer programsexecuting on one or more programmable computing devices comprising atleast one processor, a data storage component (including volatile memoryor non-volatile memory or other data storage elements or a combinationthereof) and at least one communication interface.

For example and without limitation, the programmable computers (referredto below as computing devices) may be a server, network appliance,embedded device, computer expansion module, a personal computer, laptop,personal data assistant, cellular telephone, smart-phone device, tabletcomputer, a wireless device or any other computing device capable ofbeing configured to carry out the methods described herein.

In some embodiments, the communication interface may be a networkcommunication interface. In embodiments in which elements are combined,the communication interface may be a software communication interface,such as those for inter-process communication (IPC). In still otherembodiments, there may be a combination of communication interfacesimplemented as hardware, software, and combination thereof.

In some embodiments, each program may be implemented in a high levelprocedural or object-oriented programming and/or scripting language tocommunicate with a computer system. However, the programs can beimplemented in assembly or machine language, if desired. In any case,the language may be a compiled or interpreted language.

Program code may be applied to input data to perform the functionsdescribed herein and to generate output information. The outputinformation is applied to one or more output devices, in known fashion.

Each program may be implemented in a high level procedural or objectoriented programming and/or scripting language, or both, to communicatewith a computer system. However, the programs may be implemented inassembly or machine language, if desired. In any case, the language maybe a compiled or interpreted language. Each such computer program may bestored on a storage media or a device (e.g. ROM, magnetic disk, opticaldisc) readable by a general or special purpose programmable computer,for configuring and operating the computer when the storage media ordevice is read by the computer to perform the procedures describedherein.

In some embodiments, the systems and methods as described herein mayalso be implemented as a non-transitory computer-readable storage mediumconfigured with a computer program, wherein the storage medium soconfigured causes a computer to operate in a specific and predefinedmanner to perform at least some of the functions as described herein.

Furthermore, the systems, processes and methods of the describedembodiments are capable of being distributed in a computer programproduct comprising a computer readable medium that bears computer usableinstructions for one or more processors. The medium may be provided invarious forms, including one or more diskettes, compact disks, tapes,chips, wireline transmissions, satellite transmissions, internettransmission or downloadings, magnetic and electronic storage media,digital and analog signals, and the like. The computer useableinstructions may also be in various forms, including compiled andnon-compiled code.

The various embodiments described herein generally relate to methods(and associated systems configured to implement the methods) forrepresenting usage of an electronic learning system. As noted, theelectronic learning system can collect and manage data associated withits usage. In traditional learning environments, such as a classroomsetting, instructors cannot readily determine how, or even whether, thecourse materials are used by the students. As a result, instructorstypically rely on voluntary surveys issued to the students at the end ofthe course to receive feedback concerning the course content or courseassessments to estimate the value of the course contents.

These sources of information in traditional learning environments arerather limited. The survey and course assessments are typicallycompleted near the end of the course and therefore instructors cannotadjust the course materials during the pendency of the course, ifneeded. Also, the data from the surveys can be overly subjective and mayalso be incomplete since not all students may submit responses to asurvey. Although a student's results from course assessments maygenerally be indicative of that student's understanding of the coursematerials, the results cannot be properly correlated with the quality ofthe course content since it is difficult, in a traditional learningenvironment, to determine how, or whether, that student used the coursematerials.

Electronic learning systems, on the other hand, can collect and trackdata associated with its usage. The methods and systems described hereinare directed to representing that usage. An example method includesreceiving an input that is indicative of a selection of a property ofthe electronic learning system and representation parameters that definea scope of the usage to be represented. The property can include acourse content that is provided by the electronic learning system and anactivity (e.g., some form of interaction with the electronic learningsystem) available for that course content. The representation parameterscan include a period of interest, for example.

Based on the property and the representation parameters, the electroniclearning system can generate a usage indicator for that property usingdata available for that property during that period of interest. Forexample, the usage indicator can be generated based on an event countstored at the storage components of the electronic learning system forthat property. The usage indicator can generally be reflective of ausage amount of that property during the period of interest.

The electronic learning system can then generate visualizations, orgraphical representations, to illustrate the usage indicator so that therelevant user, such as the instructor of that course, can easilyunderstand how that property is used and the value of that property infacilitating the students' learning.

Referring now to FIG. 1 , illustrated therein is a schematic diagram 10of components interacting with an electronic learning system 30 forproviding electronic learning according to some embodiments.

As shown in the schematic diagram 10, one or more users 12, 14 mayaccess the electronic learning system 30 to participate in, create, andconsume electronic learning services, including educational content suchas courses. In some cases, the electronic learning system 30 may be partof (or associated with) a traditional “bricks and mortar” educationalinstitution (e.g. a grade school, university or college), another entitythat provides educational services (e.g. an online university, a companythat specializes in offering training courses, an organization that hasa training department, etc.), or may be an independent service provider(e.g. for providing individual electronic learning).

It should be understood that a course is not limited to formal coursesoffered by formal educational institutions. The course may include anyform of learning instruction offered by an entity of any type. Forexample, the course may be a training seminar at a company for a groupof employees or a professional certification program (e.g. ProjectManagement Professional™ (PMP), Certified Management Accountants (CMA),etc.) with a number of intended participants.

In some embodiments, one or more educational groups 16 can be defined toinclude one or more users 12, 14. For example, as shown in FIG. 1 , theusers 12, 14 may be grouped together in the educational group 16. Theeducational group 16 can be associated with a particular course (e.g.History 101 or French 254, etc.), for example. The educational group 16can include different types of users. A first user 12 can be responsiblefor organizing and/or teaching the course (e.g. developing lectures,preparing assignments, creating educational content, etc.), such as aninstructor or a course moderator. The other users 14 can be consumers ofthe course content, such as students.

In some examples, the users 12, 14 may be associated with more than oneeducational group 16 (e.g. some users 14 may be enrolled in more thanone course, another example user 12 may be a student enrolled in onecourse and an instructor responsible for teaching another course, afurther example user 12 may be responsible for teaching several courses,and so on).

In some examples, educational sub-groups 18 may also be formed. Forexample, the users 14 shown in FIG. 1 form an educational sub-group 18.The educational sub-group 18 may be formed in relation to a particularproject or assignment (e.g. educational sub-group 18 may be a lab group)or based on other criteria. In some embodiments, due to the nature ofelectronic learning, the users 14 in a particular educational sub-group18 may not need to meet in person, but may collaborate together usingvarious tools provided by the electronic learning system 30.

In some embodiments, other educational groups 16 and educationalsub-groups 18 could include users 14 that share common interests (e.g.interests in a particular sport), that participate in common activities(e.g. users that are members of a choir or a club), and/or have similarattributes (e.g. users that are male, users under twenty-one years ofage, etc.).

Communication between the users 12, 14 and the electronic learningsystem 30 can occur either directly or indirectly using any one or moresuitable computing devices. For example, the user 12 may use a computingdevice 20 having one or more device processors such as a desktopcomputer that has at least one input device (e.g. a keyboard and amouse) and at least one output device (e.g. a display screen andspeakers).

The computing device 20 can generally be any suitable device forfacilitating communication between the users 12, 14 and the electroniclearning system 30. For example, the computing device 20 could bewirelessly coupled to an access point 22 (e.g. a wireless router, acellular communications tower, etc.), such as a laptop 20 a, awirelessly enabled personal data assistant (PDA) or smart phone 20 b, atablet computer 20 d, or a game console 20 e. The computing device 20could be coupled to the access point 22 over a wired connection 23, suchas a computer terminal 20 c.

The computing devices 20 may communicate with the electronic learningsystem 30 via any suitable communication channels.

The computing devices 20 may be any networked device operable to connectto the network 28. A networked device is a device capable ofcommunicating with other devices through a network such as the network28. A network device may couple to the network 28 through a wired orwireless connection.

As noted, these computing devices may include at least a processor andmemory, and may be an electronic tablet device, a personal computer,workstation, server, portable computer, mobile device, personal digitalassistant, laptop, smart phone, WAP phone, an interactive television,video display terminals, gaming consoles, and portable electronicdevices or any combination of these. These computing devices may behandheld and/or wearable by the user.

In some embodiments, these computing devices may be a laptop 20 a, or asmartphone device 20 b equipped with a network adapter for connecting tothe Internet. In some embodiments, the connection request initiated fromthe computing devices 20 a, 20 b may be initiated from a web browser anddirected at the browser-based communications application on theelectronic learning system 30.

For example, the computing devices 20 may communicate with theelectronic learning system 30 via the network 28. The network 28 mayinclude a local area network (LAN) (e.g., an intranet) and/or anexternal network (e.g., the Internet). For example, the computingdevices 20 may access the network 28 by using a browser applicationprovided on the computing device 20 to access one or more web pagespresented over the Internet via a data connection 27.

The network 28 may be any network capable of carrying data, includingthe Internet, Ethernet, plain old telephone service (POTS) line, publicswitch telephone network (PSTN), integrated services digital network(ISDN), digital subscriber line (DSL), coaxial cable, fiber optics,satellite, mobile, wireless (e.g. Wi-Fi, WiMAX), SS7 signaling network,fixed line, local area network, wide area network, and others, includingany combination of these, capable of interfacing with, and enablingcommunication between the computing devices 20 and the electroniclearning system 30, for example.

In some examples, the electronic learning system 30 may authenticate anidentity of one or more of the users 12, 14 prior to granting the user12, 14 access to the electronic learning system 30. For example, theelectronic learning system 30 may require the users 12, 14 to provideidentifying information (e.g., a login name and/or a password) in orderto gain access to the electronic learning system 30.

In some examples, the electronic learning system 30 may allow certainusers 12, 14, such as guest users, access to the electronic learningsystem 30 without requiring authentication information to be provided bythose guest users. Such guest users may be provided with limited access,such as the ability to review one or more components of the course todecide whether they would like to participate in the course but withoutthe ability to post comments or upload electronic files.

In some embodiments, the electronic learning system 30 may communicatewith the access point 22 via a data connection 25 established over theLAN. Alternatively, the electronic learning system 30 may communicatewith the access point 22 via the Internet or another external datacommunications network. For example, one user 14 may use the laptop 20 ato browse to a webpage (e.g. a course page) that displays elements ofthe electronic learning system 30.

The electronic learning system 30 can include one or more components forproviding electronic learning services. It will be understood that insome embodiments, each of the one or more components may be combinedinto fewer number of components or may be separated into furthercomponents. Furthermore, the one or more components in the electroniclearning system 30 may be implemented in software or hardware, or acombination of software and hardware.

For example, the electronic learning system 30 can include one or moreprocessing components, such as computing servers 32. Each computingserver 32 can include one or more processor. The processors provided atthe computing servers 32 can be referred to as “system processors” whileprocessors provided at computing devices 20 can be referred to as“device processors”. The computing servers 32 may be a computing device20 (e.g. a laptop or personal computer).

It will be understood that although two computing servers 32 are shownin FIG. 1 , one or more than two computing servers 32 may be provided.The computing servers 32 may be located locally together, or distributedover a wide geographic area and connected via the network 28.

The system processors may be configured to control the operation of theelectronic learning system 30. The system processors can initiate andmanage the operations of each of the other components in the electroniclearning system 30. The system processor may also determine, based onreceived data, stored data and/or user preferences, how the electroniclearning system 30 may generally operate.

The system processor may be any suitable processors, controllers ordigital signal processors that can provide sufficient processing powerdepending on the configuration, purposes and requirements of theelectronic learning system 30. In some embodiments, the system processorcan include more than one processor with each processor being configuredto perform different dedicated tasks.

In some embodiments, the computing servers 32 can transmit data (e.g.electronic files such as web pages) over the network 28 to the computingdevices 20. The data may include electronic files, such as webpages withcourse information, associated with the electronic learning system 30.Once the data is received at the computing devices 20, the deviceprocessors can operate to display the received data.

The electronic learning system 30 may also include one or more datastorage components 34 that are in electronic communication with thecomputing servers 32. The data storage components 34 can include RAM,ROM, one or more hard drives, one or more flash drives or some othersuitable data storage elements such as disk drives, etc. The datastorage components 34 may include one or more databases, such as arelational database (e.g., a SQL database), for example.

The data storage components 34 can store various data associated withthe operation of the electronic learning system 30. For example, coursedata 35, such as data related to a course's framework, educationalcontent, and/or records of assessments, may be stored at the datastorage components 34. The data storage components 34 may also storeuser data, which includes information associated with the users 12, 14.The user data may include a user profile for each user 12, 14, forexample. The user profile may include personal information (e.g., name,gender, age, birthdate, contact information, interests, hobbies, etc.),authentication information to the electronic learning system 30 (e.g.,login identifier and password) and educational information (e.g., whichcourses that user is enrolled in, the user type, course contentpreferences, etc.).

The data storage components 34 can store authorization criteria thatdefine the actions that may be taken by certain users 12, 14 withrespect to the various educational contents provided by the electroniclearning system 30. The authorization criteria can define differentsecurity levels for different user types. For example, there can be asecurity level for an instructing user who is responsible for developingan educational course, teaching it, and assessing work product from thestudent users for that course. The security level for those instructingusers, therefore, can include, at least, full editing permissions toassociated course content and access to various components forevaluating the students in the relevant courses.

In some embodiments, some of the authorization criteria may bepre-defined. For example, the authorization criteria can be defined byadministrators so that the authorization criteria are consistent for theelectronic learning system 30, as a whole. In some further embodiments,the electronic learning system 30 may allow certain users, such asinstructors, to vary the pre-defined authorization criteria for certaincourse contents.

The electronic learning system 30 can also include one or more backupservers 31. The backup server can store a duplicate of some or all ofthe data 35 stored on the data storage components 34. The backup server31 may be desirable for disaster recovery (e.g. to prevent data loss inthe event of an event such as a fire, flooding, or theft). It should beunderstood that although only one backup server 31 is shown in FIG. 1 ,one or more backup servers 31 may be provided in the electronic learningsystem 30. The one or more backup servers 31 can also be provided at thesame geographical location as the electronic learning system 30, or oneor more different geographical locations.

The electronic learning system 30 can include other components forproviding the electronic learning services. For example, the electroniclearning system 30 can include a management component that allows users12, 14 to add and/or drop courses and a communication component thatenables communication between the users 12, 14 (e.g., a chat software,etc.). The communication component may also enable the electroniclearning system 30 to benefit from tools provided by third-partyvendors. Other example components will be described with reference toFIG. 2 .

Referring now to FIG. 2 , which is a block diagram 100 of somecomponents 5 that may be implemented in the electronic learning system30 according to some embodiments. In the example of FIG. 2 , the variousillustrated components are provided at one of the computing servers 32.

As shown in FIG. 2 , the computing server 32 may include a systemprocessor 110, an interface component 120, a local storage component 130and a representation component 140. Each of the system processor 110,the interface component 120, the local storage component 130 and therepresentation component 140 can be in electronic communication with oneanother. It should be noted that in alternative embodiments, the systemprocessor 110, the interface component 120, the local storage component130 and the representation component 140 may be combined or may beseparated into further components. Furthermore, the system processor110, the interface component 120, the local storage component 130 andthe representation component 140 may be implemented using software,hardware or a combination of both software and hardware.

Generally, the system processor 110 controls the operation of thecomputing server 32 and, as a result, various operations of theelectronic learning system 30. For example, the system processor 110 maybe configured to initiate the representation component 140 to generate arepresentation of the usage of the electronic learning system 30 inaccordance with the methods described herein.

The interface component 120 may be any interface that enables thecomputing server 32 to communicate with the other computing servers 32,backup servers 31 and data storage components 34 within the electroniclearning system 30. The interface component 120 may also include anyinterface that enables the computing server 32 to communicate withthird-party systems. In some embodiments, the interface component 120can include at least one of a serial port, a parallel port or a USBport. The interface component 120 may also include at least one of anInternet, Local Area Network (LAN), Ethernet, Firewire, modem or digitalsubscriber line connection. Various combinations of these elements maybe incorporated within the interface component 120.

In some embodiments, the interface component 120 may receive input fromthe computing devices 20 via various input components, such as a mouse,a keyboard, a touch screen, a thumbwheel, a track-pad, a track-ball, acard-reader, voice recognition software and the like depending on therequirements and implementation of the electronic learning system 30.

The local storage component 130 may be provided at the computing server32 for temporary storage of data associated with various operations ofthe system 10 processor 110. The local storage component 130 may receivedata from and/or transmit data to the data storage components 34.

The representation component 140 can include the software and dataassociated with the various methods for generating a representation ofthe usage of the electronic learning system 30 as described herein. Thesystem processor 110 can initiate operation of the representationcomponent 140 for representing the usage of the electronic learningsystem 30, for example. Example embodiments will now be described withreference to FIGS. 3 to 6C.

Referring now to FIG. 3 , a flowchart diagram illustrating an examplemethod 200 for representing usage of the electronic learning system 30is shown. To illustrate the method 200, reference will be madesimultaneously to FIGS. 4A to 6C.

At 210, the system processor 110 receives an input indicative of aselection of a property of the electronic learning system 30 to berepresented.

As noted, the property can include a course content that is provided bythe electronic learning system 30 and an activity available for thatcourse content. Example properties will now be described with referenceto FIG. 4A.

FIG. 4A illustrates a screenshot 300A of an example user interface 310Afor receiving an input for selecting the property in accordance with anexample embodiment. The user interface 310A is displayed in an exampleweb browser 302. The user interface 310A includes a course contentselection component 320 and an activity selection component 330. It willbe understood that the selection components 320, 330 are only exampleinterface components for receiving inputs from the users 12, 14 and thatother types of interface components may be used for receiving data.

The course content can generally include various aspects of a course,such as a discussion forum, a learning objective page, a chat, a messageboard, a Wiki page, an assessment (e.g., a self-assessment, quiz,mid-term exam, etc.), a rubric, an assignment, a survey, a checklist,and a submission tool. As will be described with reference to FIG. 4A,the course contents can be organized into different modules, such as bya topic. It is possible that multiple of the same type of coursecontents is provided for a course and within one particular collection.

The various course contents may also be organized into variouscategories by the system processor 110, in some embodiments. Thecategories may be based on certain frequencies of interaction (e.g., thecourse content with the most viewed forum, the least commented chat, thereview that received the greatest number of ratings or the content thatwas bookmarked the most) or a range of values (e.g., the user who earnedthe highest grade for an assessment, with the fewest number of repliesto discussion forums or with a highest average rating on a comment).Other types of categories can include course contents that received someform of acknowledgement (e.g., viewed, received a rating, received areview, was shared onto a social media platform, was bookmarked, wasprinted, etc.), course contents associated with an activity thatexceeded a predefined amount (e.g., course content on which one or morestudent spent more than a predefined period of time, course content thatreceived more than a predefined number of feedbacks, etc.) and otherrelevant categories.

In the example of FIG. 4A, the relevant courses are “Math 101” and“Calculus II”. For each course, a list of course content available forselection is shown, that is a course content list 322 is available forMath 101 and a course content list 324 is available for Calculus II.

In the course content list 322, the course contents available for Math101 are organized into three collections, namely a “Module 1” collection(a first collection 322 a), a “Module 2” collection (a second collection322 b) and an exam (a third collection 322 c). First and secondcollections 322 a and 322 b include other course contents. The firstcollection 322 a includes an introduction content providing an overviewof Math 101, a prerequisites content indicating the requirements forenrolling into Math 101 and an evaluation criteria content providing therubric for Math 101. The second collection 324 b includes a coursecontent on the topic of geometry and a self-assessment content (coursecontent 326).

Similarly, the course contents available for Calculus II are organizedinto three collections, namely a “Module 1” collection (a firstcollection 324 a), a “Module 2” collection (a second collection 324 b)and an exam (a third collection 324 c). First and second collections 324a and 324 b include other course contents. The first collection 324 aincludes the same type of course contents as the first collection 322 abut the course contents are specific to Calculus II. The secondcollection 324 b includes a course content on the topic of integrationand a self-assessment content (course content 328).

In the illustrated example of FIG. 4A, the same type of course contents,namely the self-assessment contents 326, 328 in the respective secondcollections 322 b and 324 b, and the exams 322 c and 324 c are selectedby the user 12, 14 to be represented. It will be understood that thesame type of course contents is shown as selected in FIG. 4A for ease ofexposition and that different types of course contents may be selectedinstead.

Continuing with reference to FIG. 4A, various different activities areshown in the activity selection component 330.

Generally, the activities available for the course contents can becategorized based on the type of the interaction. Example categories forthe activities can include user interactions, social interactions,system contributions and user characteristics.

The user interactions can include actions conducted by a student withrespect to the electronic learning system 30. For example, the userinteractions can include an amount of time spent viewing a coursecontent, an amount of time spent responding to a course content, anumber of revisions made to a course content, a number of keystrokesinvolved with a course content, a rating provided for a course content,a review for a course content and aspects of that review (e.g., aquality, a length of the review and an amount of time spent writing thatreview), and a comment provided for a course content (e.g., a quality, alength of the comment and an amount of time spent writing that comment),a number of completions of the course content, a number of contributionsto a course content, whether the course content was created by thatstudent, whether a course content was bookmarked and whether a coursecontent was printed.

The social interactions can include electronic interactions betweenstudents and interactions on social media platforms by a student or byanother social media contributor associated with the student. The socialmedia platforms may include Twitter™, Facebook™, Google+™ and othersimilar social media tools.

Example social media interactions can include generating a social mediamessage that is associated with a course content, a reply to that socialmedia message by another social media contributor, sharing of thatsocial media message by another social media contributor, receiving afeedback to that social media message from another social mediacontributor, and providing a feedback to the social media message or areceived feedback.

Example electronic interactions between students can include receivingor providing a peer review to a student-generated course content,receiving a response to a question posted on a course content (e.g.,discussion forum), any actions associated with the response (e.g., theresponse is “flagged” or “tagged”), being referenced by another studentand having a contribution referenced by another student.

The system contributions can include a grade achieved, whether anactivity was assessed, whether a goal was achieved, whether a curriculumtarget was met, and whether a learning objective was met.

The user characteristics include different aspects associated with theusers 12, 14 such as a number of courses a student is enrolled in, asemester level achieved, the enrolled program (e.g., the Faculty andprogram of study, etc.), a student's current grade, a student's overallgrade (e.g., the student's grade point average across multiple courses,etc.) and any associated educational groups 16 or sub-groups 18 (e.g.,course-based, project-based, user characteristics, user-defined,instructor defined, etc.). Other user characteristics can include alocation of the user, such as a geographic location (e.g., country,city, etc.), Global Positioning System (GPS) coordinates, an altitudelevel or a relative position from a landmark (e.g., a Universitybuilding). For example, the property can include an indication of aspecific educational group 16 for which the usage of the electroniclearning system 30 is considered.

Referring still to FIG. 4A, as shown in the activity selection component330, various different activities may be selected for each coursecontent in the course content lists 322 and 324. The example activitiesin FIG. 4A include a number of completions for the course content(activity 330 a), time spent being over an hour (activity 330 b), numberof revisions (activity 330 c), whether the course content was shared ona social network (activity 330 d), whether the course content wasprinted (activity 330 e) or bookmarked (activity 330 f). The activitieslisted in the activity selection component 330 are merely examples andit will be understood that fewer or a greater number of activities maybe represented by the electronic learning system 30.

In the example embodiment of FIG. 4A, the activities selected are theactivities 330 a and 330 b. Accordingly, the input received by thesystem processor 110 in this example indicates that the property to berepresented includes the number of completions and number of time spenton the course content being over one hour for each of theself-assessments 326 and 328 and the exams 322 c and 324 c.

The user interface 310A shown in FIG. 4A further includes a pushbutton304 for receiving an input from the user 12, 14 for proceeding to thenext step in the described method for representing the usage based onthe selected property. For this example, the system processor 110proceeds to display the user interface 310B shown in FIG. 4B in responseto receiving an input selecting the pushbutton 304. It will beunderstood that the illustrated pushbutton 304 is merely an example andthat other user interface icons may similarly be used. Alternatively,the user interface 310A may be configured to include all the interfacecomponents shown in both user interfaces 310A 25 and 310B.

At 220, the system processor 110 receives representation parameters thatdefine a scope of the usage to be represented.

The representation parameters can vary the scope of the usage to berepresented.

FIG. 4B illustrates a screenshot 300B of an example user interface 310Bfor receiving the representation parameters in accordance with anexample embodiment. In the example shown in FIG. 4B, the scope is onlylimited based on time. It will be understood that different oradditional types of scope variations may similarly be applied.

The user interface 310B includes a usage scope component 340 forreceiving the representation parameters from the user 12, 14. The usagescope component 340 includes a start date component 342 for receiving astart date for the period of interest and an end date component 344 forreceiving an end date for the period of interest. The period of interestis defined by the start date and the end date. In this example, theperiod of interest begins on Nov. 11, 2013 and ends on Dec. 15, 2013.Once the period of interest is received by the system processor 110, thesystem processor 110 can identify the relevant data for the propertyduring that time.

Also provided in the usage scope component 340 is a time intervalcomponent 346. In some embodiments, the system processor 110 can furtherorganize the data into segments based on the time interval received viathe time interval component 346. The time interval is shorter than thereceived period of interest and may be any time periods, such as acertain period of time (e.g., 10 minutes), days (e.g., a week, a day,etc.), months and other similar time periods. By segmenting the data,the user 12, 14 can more clearly identify the periods of interest forreview and analysis. For example, a time interval that is 10 minuteslong can be useful for understanding usage of the electronic learningsystem 30 right before a significant course assessment, such as a quiz,whereas a time interval that is 1 year can be useful for understandingsemester-wide trends.

In some embodiments, the time interval can be a specific portion withinthe period of interest, as selected by the user 12, 14.

It will be understood that the start date component 342, the end datecomponent 344 and the time interval component 346 shown in FIG. 4B aremerely examples and that other user interface components, such as aslider bar, may similarly be used.

In some embodiments, the scope of the usage representation can vary by30 educational groups 16. As noted, the educational groups 16 may bedefined based on different aspects of the electronic learning system 30,such as user properties and course enrollment. By targeting certaineducational groups 16, the usage representation can be generated so thatthe user 12, 14 can develop a better understanding of the correlationbetween the course contents and the results of the assessments. Forexample, the educational groups 16 can be defined by the instructor toinclude students with a similar current course grade average, a similaroverall grade point average, students enrolled in a particular facultyand/or program of study, or students with a similar score in aparticular assessment. For each of these educational groups 16, theinstructor can determine how those students used the course contents andwhether the course contents affect the results achieved by thosestudents.

The user interface 310B shown in FIG. 4B further includes a pushbutton306 for receiving an input from the user 12, 14. For this example, thesystem processor 110 proceeds to generate a representation (such as thatshown in FIG. 5D or FIG. 6C, for example) of the usage of the electroniclearning system 30 based on the inputs received via the user interfaces310A and 310B. Similar to the pushbutton 304, it will be understood thatthe illustrated pushbutton 306 is merely an example and that other userinterface icons may similarly be used.

At 230, the system processor 110 determines an event count for theproperty during the period of interest.

Generally, the event count can include the number of events associatedwith the property selected at 210 for the period of interest identifiedat 220. The event count can be determined from the number of relevantevents that are stored at the data storage components 34. The eventcount may be determined by the system processor 110 or provided to thesystem processor 110 by the data storage components 34. In someembodiments, the relevant events can first be retrieved from the datastorage components 34 and stored at the local storage component 130 forfacilitating the determination of the event count by the systemprocessor 110.

The system processor 110 may, in some embodiments, determine the eventcount by first clearing the event count stored at the data storagecomponents 34 or the local storage component 130. For example, thesystem processor 110 can set the event count to zero. The systemprocessor 110 can then retrieve, from the data storage components 34 orthe local storage component 130, events that satisfy the representationparameters identified in FIG. 4B. For each retrieved event, the systemprocessor 110 then determines whether that retrieved event is associatedwith the property identified in FIG. 4A. In the case that the retrievedevent is associated with the property, the system processor 110 proceedsto increment the event count.

Other example methods of determining the event counts will be describedwith reference to FIGS. 5A to 5C. For ease of exposition, only the eventcount for the self-assessment content 326 for Math 101 is shown in FIGS.5A to 5C. Methods for determining the event counts for two differentcourses will be described with reference to FIGS. 6A to 6C.

FIG. 5A is a table 400A illustrating the event counts for the propertyselected at 210 in respect of the self-assessment content 326 during theidentified period of interest, in accordance with an example embodiment.

As shown in FIG. 5A, the period of interest is separated into a set ofsub-periods based on the time interval selected in FIG. 4B. Each columnin table 400A corresponds to a sub-period within the period of interest,namely Nov. 11, 2013 to Dec. 15, 2013. In the case of a semester course,for example, the identified period of interest generally corresponds toa time period closer to the date of final exams. The usage of theself-assessment 326 provided by the electronic learning system 30 forMath 101 can help the instructor of Math 101 determine how and when theself-assessment 326 is used during that time period.

For embodiments in which the period of interest is not separable intosub-periods that each corresponds to the time interval, the period ofinterest is separated into at least one sub-period that corresponds tothe time interval.

For each sub-period, the system processor 110 can determine a count foreach activity identified in FIG. 4A (e.g., activities 330 a and 330 b)for the self-assessment content 326 during that sub-period. The eventcount will be a sum of the counts from each sub-period. As shown in FIG.5A, the activity 330 a is associated with the counts 410 a to 410 e andthe activity 330 b is associated with the counts 420 a to 420 e.

From FIG. 5A, it can be seen, from the counts 410 a to 410 e, that thenumber of completions increases substantially as the final examsapproaches and, from the counts 420 a to 420 e, it can be seen that thetime spent on the self-assessment 326 fluctuates.

The counts 410 a to 410 e and 420 a to 420 e shown in FIG. 5A are rawdata values that directly correspond to the number of relevant eventsstored at the data storage components 34. The counts 410 a to 410 e and420 a to 420 e may be further processed by the system processor 110 inorder to demonstrate the correlation between the various data sets andto generate the usage indicator.

To prevent irregular skews in the counts 410 a to 410 e and 420 a to 420e, the system processor 110 may adjust the counts 410 a to 410 e and 420a to 420 e base on a count parameter. The count parameter may includeone or more limitations for the counts 410 a to 410 e and 420 a to 420e. For instance, the system processor 110 can operate to limit thecounts 410 a to 410 e and 420 a to 420 e by an upper count limit, alower count limit or both.

The count parameter can help the system processor 110 analyze the dataand to smooth the data. For example, with the count parameter, thesystem processor 110 can analyze the counts 410 a to 410 e and 420 a to420 e to remove any irregular counts in order to more clearly illustratetrends in the usage of the electronic learning system 30. In this way,the resulting usage representation would not be skewed by irregularactivities. For example, an instructor may be interested in identifyingthe most popular topics within the course and define topics to be“popular” if there are at least ten interactions with the coursecontents associated with that topic. In this way, the system processor110 can eliminate any topic with fewer than ten interactions in order toprovide the user 12, 14 with the data of interest.

To restrict the counts 410 a to 410 e and 420 a to 420 e by the uppercount limit, the system processor 110 can, in some embodiments, receivean upper count limit as part of the count parameter and operate toadjust each count by determining whether the counts 410 a to 410 e and420 a to 420 e exceeds the upper count limit.

The system processor 110 can reduce the count to the upper count limitif that count exceeds the upper count limit. With respect to the counts410 a to 410 e and 420 a to 420 e shown in FIG. 5A, if the upper countlimit is ‘20’, the system processor 110 will operate to reduce the count410 e to ‘20’. No other counts in FIG. 5A require adjustment. It will beunderstood that other upper count limits can similarly be applieddepending on the requirements of the usage representation to begenerated by the system processor 110

In some embodiments, the system processor 110 can receive the lowercount limit as part of the count parameter and operate to adjust eachcount based on the lower count limit. For each count, the systemprocessor 110 can generate an intermediary count by subtracting thelower count limit from the count. If the resulting intermediary count isless than zero, the system processor 110 can then set that count to zerobut otherwise, the system processor 110 can use the intermediary eventcount as the event count.

Referring again to the counts 410 a to 410 e and 420 a to 420 e shown inFIG. 5A, if the lower count limit is ‘5’, the system processor 110 willoperate to vary the counts 410 a to 410 e and 420 a to 420 eaccordingly. For the activity 330 a, the count 410 a will remain at ‘0’since the intermediary count for Week 1 is less than zero; the count 410b will also be ‘0’ since the intermediary count for Week 2 is less thanzero; the count 410 c will be ‘10’, which is the intermediary count forWeek 3; the count 410 d will be ‘15’, which is the intermediary countfor Week 4; and the count 410 e will be ‘25’, which is the intermediarycount for Week 5.

Similarly, for the activity 330 b, the count 420 a will remain at ‘0’since the intermediary count for Week 1 is less than zero; the count 420b will also be ‘0’ since the intermediary count for Week 2 is less thanzero; the count 420 c will be ‘0’, which is the intermediary count forWeek 3; the count 420 d will be ‘3’, which is the intermediary count forWeek 4; and the count 420 e will be ‘0’, which is the intermediary countfor Week 5.

It will be understood that other lower count limits can similarly beapplied depending on the requirements of the usage representation to begenerated by the system processor 110.

In some examples, the system processor 110 can apply various adjustmentsto the counts, either the raw data values or corresponding values thathave been limited based on the count parameter, in order to normalizethe values. Normalizing the counts 410 a to 410 e and 420 a to 420 e isused to generate unit-less values between 0.0 and 1.0 so that theresulting values can be graphically represented accordingly.

Referring now to FIG. 5B, which is a table 400B illustrating adjustedevent counts 410 a′ to 410 e′ and 420 a′ to 420 e′ for each of thecounts 410 a to 410 e and 420 a to 420 e shown in FIG. 5A in accordancewith an example embodiment.

In some embodiments, the system processor 110 can adjust the counts foreach activity based on the count with the greatest value. For example,the system processor 110 can identify the top count from the counts 410a to 410 e for the activity 330 a in FIG. 5A, namely the count 410 e.The system processor 110 can then normalize each of the counts 410 a to410 e for the activity 330 a using the count 410 e. The adjusted eventcounts 410 a′ to 410 e′ shown in FIG. 5B for the activity 330 acorrespond to normalized counts.

For the activity 330 b, the system processor 110 can also identify thetop count, namely the count 420 d. The system processor 110 can thennormalize each of the counts 420 a to 420 e for the activity 330 b usingthe count 420 d. The adjusted event counts 420 a′ to 420 e′ shown inFIG. 5B for the activity 330 b correspond to normalized counts.

As shown in FIG. 5B, the adjusted event counts 410 a′ to 410 e′ and 420a′ to 420 e′ generated by normalizing the counts 410 a to 410 e and 420a to 420 e fall within a range of 0.0 to 1.0.

In some further embodiments, the system processor 110 may apply furtheradjustments to the counts 410 a to 410 e and 420 a to 420 e shown inFIG. 5A or to the adjusted event counts 410 a′ to 410 e′ and 420 a′ to420 e′ shown in FIG. 5B in accordance with a transformation modeindicator. By further transforming the adjusted event counts 410 a′ to410 e′ and 420 a′ to 420 e′, the resulting values can be aggregated withdifferent methods. The transformation of the counts can facilitatefurther smoothing of the data and illustration of different trends inthe data.

The system processor 110 may receive an indication of a particulartransformation mode indicator via one of the user interfaces 310A or310B, for example. The transformation mode indicator generally providesa type of adjustment to be applied to the counts 410 a to 410 e and 420a to 420 e. The various transformation modes can include a standardmode, a cumulative average mode and a peak mode. Other transformationmodes may similarly be applied.

When operating in the standard mode, the system processor 110 does notvary the counts 410 a to 410 e and 420 a to 420 e for each of thesub-periods. Instead, the counts 410 a to 410 e and 420 a to 420 econtinue to correspond to the aggregated values during that time period.The current mode may be the default mode for some embodiments.

When operating in the peak mode, the system processor 110 operates toidentify the sub-period associated with the top count for a particularactivity. The top count is the count with the highest value. The peakmode can be particularly useful for illustrating trends in the variousactivities. After identifying that sub-period, the system processor 110then replaces the counts in the subsequent sub-periods with that topcount. For example, for the self-assessment 326 in Math 101, the topcount for the activity 330 b is the adjusted event count 420 d′. Thesystem processor 110 will therefore replace the adjusted event count 420e′ with the adjusted event count 420 d′.

In some embodiments, a count may need to meet additional requirements(other than having the highest value for a particular activity) beforebeing considered the top count. For example, a count can be identifiedby the system processor 110 as the top count when it has the highestvalue for a particular activity and the subsequent counts do not varyfrom that highest value for more than a certain portion of that highestvalue for a certain period of time. That is, with respect to the exampleshown in FIG. 5B, although the adjusted event count 420 d′ is thehighest value for the activity 330 b, the adjusted event count 420 d′may only qualify as the top count if the subsequent counts (e.g.,adjusted event count 420 d′) do not vary from the adjusted event count420 d′ by more than 10% of the value of the adjusted event count 420 d′.That is, the adjusted event count 420 e′ needs to be at least 0.9 inorder for the adjusted event count 420 d′ to qualify as the top count.Since the adjusted event count 420 e′ is less than 0.9 in the exampleshown in FIG. 5B, the adjusted event count 420 d′ would not qualify asthe top count in this case. Different percentages of the highest valueand different subsequent time periods may similarly be used foridentifying the top count.

When operating in the cumulative average mode, the system processor 110replaces the count for each sub-period with an average of the counts forthe preceding sub-periods. The cumulative average mode can also beuseful for illustrating trends in the various activities. For eachsub-period, the system processor 110 retrieves the counts from thepreceding sub-periods. The system processor 110 can then generate anadjusted event count by determining an average of the counts from thatsub-period and the preceding sub-periods. The system processor 110replaces the count for that sub-period with the adjusted event count.

For example, referring again to FIG. 5B, when the transformation mode isthe cumulative average mode, the system processor 110 operates totransform the adjusted event counts 410 a′ to 410 e′ and 420 a′ to 420e′ accordingly. For the activity 330 a, the adjusted event count 410 a′is unchanged since there is no preceding sub-period; the adjusted eventcount 410 b′ becomes ‘0.05’, which is the average of the adjusted eventcounts 410 a′ and 410 b′; the adjusted event count 410 c′ becomes ‘0.2’,which is the average of the adjusted event counts 410 a′ to 410 c′; theadjusted event count 410 d′ becomes ‘0.3175’, which is the average ofthe adjusted event counts 410 a′ to 410 d′; and the adjusted event count410 e′ becomes ‘0.454’, which is the average of the adjusted eventcounts 410 a′ to 410 e′.

For the activity 330 b, the adjusted event count 420 a′ is unchangedsince there is no preceding sub-period; the adjusted event count 420 b′becomes ‘0.065’, which is the average of the adjusted event counts 420a′ and 420 b′; the adjusted event count 420 c′ becomes approximately‘0.253’, which is the average of the adjusted event counts 420 a′ to 420c′; the adjusted event count 420 d′ becomes ‘0.44’, which is the averageof the adjusted event counts 420 a′ to 420 d′; and the adjusted eventcount 420 e′ becomes ‘0.478’, which is the average of the adjusted eventcounts 420 a′ to 420 e′.

At 240, the system processor 110 generates a usage indicator 450 for theproperty based at least on the event count.

The usage indicator 450 is generally reflective of a usage amount of theproperty during the period of interest. The usage indicator 450 can bedetermined from the normalized counts associated with the activities ofinterest and in some embodiments, with the application of weight factorsassociated with those activities. The weight factors can be anynumerical value between 0.0 to 1.0. The weight factors to be used may ormay not sum to one since it is possible that only some of the activitiesavailable for a course content is selected for representation.

In embodiments where the weight factors to be used do not sum to one,those weight factors can be proportionally adjusted to sum to one. Forinstance, if the weight factors 0.4 and 0.1 are to be used, those weightfactors can be proportionally adjusted to 0.8 and 0.2, respectively.

In some embodiments, the system processor 110 may associate differentweight factors to the different activities of the course content. Theweight factors can generally indicate an amount of influence that aparticular activity has on the usage of the electronic learning system30. The use of the weight factors allows for the different activities tobe weighted differently with respect to each other. For example, theinstructor of a course may consider one activity, such as completing aself-assessment, to be more important than another activity, such asprinting the self-assessment. The weight factors may be automaticallygenerated by the electronic learning system 30 or defined by the user 12(e.g., an instructor).

FIG. 5C is a table 400C illustrating usage indicators 450 a to 450 egenerated based on the respective adjusted event counts 410 a′ to 410 e′and 420 a′ to 420 e′ of FIG. 5B. In this example, a weight factor of 0.8is associated with the activity 330 a and a weight factor of 0.2 isassociated with the activity 330 b. The user 12 in this example,therefore, considers the activity 330 a (e.g., the number of times theself-assessment 326 was completed) to be more influential than theactivity 330 b (e.g., the number of times in which more than one hourwas spent on the self-assessment 326).

In some other embodiments, the system processor 110 may not associateany weight factors to the activities for a course content. For example,the instructor may consider the activities to have equal influence onthe usage of the electronic learning system 30. The usage indicator 450may be determined by applying the same weight factor to each of theadjusted event counts. The weight factor can correspond to an inverse ofthe total number of activities being considered for the usage indicator450. That is, referring again to FIG. 5C, instead of applying the weightfactors, 0.8 and 0.2, to the respective activities 330 a and 330 b, theweight factor 0.5 (i.e., the inverse of two) is applied to bothactivities 330 a and 330 b. This may be the default setting in someembodiments.

After determining the usage indicator 450, the system processor 110 mayprovide a visual representation of the usage indicator 450 to assist theuser 12 in understanding the usage of the electronic learning system 30.For example, the visual representation can help an instructor determinehow their course content is being used over the duration of a course,track the progress of certain educational groups 16 during a course,and/or identify course contents that are not accessed often enough towarrant the purchase cost of that course content.

In some embodiments, the visual representation may include a heat map ora line graph. It will be understood that other types of visualrepresentations that can help distinguish between the different usageindicators 450 a to 450 e can similarly be used.

The visual representation may include, in some embodiments, anadjustable representation for which the scope of the usage to bedisplayed can be modified, possibly in or close to real-time, by inputsreceived from the user 12. In some embodiments, the adjustablerepresentation can include receiving an input from the user 12, 14requesting for the relevant usage indicators 450 to be sequentiallyillustrated, as if the adjustable representation is a video display.

For example, the adjustable representation can be varied based on inputvalues provided from the user. The input values can include variouscontrol inputs, such as “play” to show the relevant usage indicators 450sequentially, “rewind” to move the display of the usage indicators 450back in time, “forward” to move the display of the usage indicators 450forward in time, “stop” or “pause” to hold the display of the usageindicators 450 at a particular point in time and other similar controls.The adjustable representation can receive other inputs to specify aparticular point in time in which the usage indicators 450 are to bedisplayed.

Referring now to FIG. 5D, which is a screenshot 400D of a graphicalrepresentation 470 of the usage of the electronic learning system 30 inaccordance with 30 an example embodiment. The graphical representation470 is a modified heat map generated based on usage indicators 450 a to450 e of FIG. 5C and shown in and shown in an example user interface460. As is known in the art, heat maps are generally graphicalrepresentations of data in which the data values are represented bycolours. For the purpose of maintaining clarity, the usage indicators450 a to 450 e are shown in the modified heat map 470 with differentdegrees of cross-hatchings and shading. As illustrated byrepresentations 480 a to 480 e, the degree of the cross-hatching andshading in the representations 480 a to 480 e generally corresponds tothe value of the respective usage indicators 450 a to 450 e. In thisexample, the heat map 470 generally illustrates how the self-assessment326 is used over the period of interest. However, the heat map 470 doesnot illustrate how the self-assessment 326 compares with respect toother course contents. Example usage indicators correlating the usage ofdifferent course contents will be described with reference to FIGS. 6Ato 6C.

FIG. 6A illustrates table 500A and 502A for event counts of two coursecontents provided by the electronic learning system 30 in accordancewith an example embodiment. In the example of FIG. 6A, the coursecontents are in respect two different courses, a first course content326 (i.e., self-assessment for Math 101) and a second course content 328(i.e., self-assessment for Calculus II).

Table 500A includes the event counts 410 a to 410 e for the activity 330a in respect of the first course content 326. As described with respectto FIG. 5A, the event counts 410 a to 410 e correspond to the usageamount of the activity 330 a for the first course content 326 during therespective sub-periods.

Table 502A includes the event counts 510 a and 510 e for the activity330 a in respect of the second course content 328. The event counts 510a to 510 e correspond to the usage amount of the activity 330 a for thesecond course content 328 during the respective sub-periods.

By representing the values provided in table 500A and 502A, the user 12can better appreciate how those properties vary over the period ofinterest and with respect to each other. That is, in the example shownin FIG. 6A, the user 12 can compare the value of the self-assessments326 and 328 for the different courses. Depending on the course type,different course contents may be required. For example, courses that aredirected to junior students may require course contents that help themto study (e.g., self-assessments, quizzes, etc.), whereas courses thatare directed to senior students may instead require course contents thatfacilitate discussion and interaction with their peers (e.g., discussionforums, chats, etc.).

Continuing with the example shown in FIG. 6A, FIG. 6B illustrates tables500B and 502B which include adjusted event counts corresponding to theevent counts shown in FIG. 6A in accordance with an example embodiment.

As described with reference to FIG. 5B, the event counts 410 a to 410 eand 510 a to 510 e can generally be normalized with the methodsdescribed herein. The system processor 110 can, in some embodiments,adjust the event counts 410 a to 410 e and 510 a to 510 e using the samevalue so that the two sets of event counts are consistently normalized.

For instance, the system processor 110 can identify, from the eventcounts 410 a to 410 e and 510 a to 510 e, a top count that is associatedwith the highest value. The system processor 110 can then normalize theevent counts 410 a to 410 e and 510 a to 510 e with the top count.

In an example embodiment in which the first course content 326 isassociated with only one event count, namely a first event count, andthe second course content 328 is associated with only one event count,namely a second event count, the system processor 110 can identify thetop count by comparing the first event count with the second eventcount. The top count is the greater of the first event count and thesecond event count.

Continuing with the example of FIG. 6A, the system processor 110 cannormalize the event counts 410 a to 410 e and 510 a to 510 e byidentifying the event count with the greatest value, which is the eventcount 410 e. The adjusted event counts 410 a′ to 410 e′ and 510 a′ to510 e′ in FIG. 6B correspond to the event counts 410 a to 410 e and 510a to 510 e, respectively, after being normalized with the event count410 e.

Further adjustments described herein may be applied to the values shownin FIGS. 6A and 6B depending on the user preferences and other relevantrequirements.

The system processor 110 can determine usage indicators 450 for each ofthe adjusted event counts 410 a′ to 410 e′ and 510 a′ to 510 e′ providedin FIG. 6B. For the illustrated example, since only one activity isbeing considered, the adjusted event counts 410 a′ to 410 e′ and 510 a′to 510 e′ can be the usage indicators 450.

Referring now to FIG. 6C, which is a screenshot 550 of a graphicalrepresentation 570 of the usage of the electronic learning system 30 inaccordance with an example embodiment. The graphical representation 570is shown in an example user interface 560.

The graphical representation 570 in this example is a line graph andincludes a first graph 572 for the first course content 326 and a secondgraph 574 for the second course content 328. Each of the first graph 572and the second graph 574 includes the values associated with therespective usage indicators 450 in FIG. 6B. It can be seen from thefirst graph 572 that the first course content 326 is more heavily usedthan the second course content 328, with the difference beingincreasingly evident as time progresses. Based on this, theinstructor(s) for Math 101 and Calculus II can determine that theself-assessment content 326 is more valuable for more junior studentssince Math 101 is a more junior class than Calculus II and theinstructor may therefore generate more similar course contents for Math101.

embodiments herein been described here by way of example only. Variousmodification and variations may be made to these example embodiments.Also, in the various user interfaces illustrated in the figures, it willbe understood that the illustrated user interface text and controls areprovided as examples only and are not meant to be limiting. Othersuitable user interface elements may be possible.

We claim:
 1. A computer-implemented method for representing usage of anelectronic learning system, the method comprising: receiving an inputindicative of a selection of an interaction with a course contentprovided by the electronic learning system; receiving representationparameters that define a scope of the usage to be represented, therepresentation parameters including at least a period of interest;determining an event count for the interaction during the period ofinterest, the event count being a number of events stored at one or morestorage components of the electronic learning system for the interactionwith the course content; determining one or more groups of users basedat least in part on a respective level of achievement of the user withrespect to a course associated with the course content; determining acorrelation between the event count and a level of achievement withrespect to the course based at least in part on the one or more groupsof users; and providing an indication of the correlation between theevent count and the level of achievement with respect to the course. 2.The method of claim 1, wherein: the representation parameters furthercomprise a time interval that is shorter than the period of interest;and determining the event count for the interaction during the period ofinterest comprises: separating the period of interest into a set ofsub-periods according to the time interval, a length of at least onesub-period corresponds to the time interval; and determining a pluralityof counts for the period of interest, each count being the number ofevents corresponding to the interaction with the course content during asub-period and the event count being a sum of the plurality of counts.3. The method of claim 1, wherein: the count parameter comprises atleast one of an upper count limit and a lower count limit; and adjustingthe event count with the count parameter comprises: when the countparameter is the upper count limit: determining whether the event countexceeds the upper count limit; and in response to determining the eventcount exceeds the upper count limit, reducing the event count to theupper count limit; and when the count parameter is the lower countlimit: generating an intermediary event count by subtracting the lowercount limit from the event count; and if the intermediary event count isless than zero, setting the event count to zero, otherwise using theintermediary event count as the event count.
 4. A computer-implementedmethod for representing usage of an electronic learning system, themethod comprising: receiving an input indicative of a selection of aproperty of the electronic learning system to be represented, theproperty including a course content provided by the electronic learningsystem and an activity available for that course content; receivingrepresentation parameters that define a scope of the usage to berepresented, the representation parameters including at least a periodof interest; determining an event count for the property during theperiod of interest, the event count corresponding to a number of eventsstored at one or more storage components of the electronic learningsystem in association with the property for the period of interest; anddetermining one or more groups of users based at least in part on arespective level of achievement of the user with respect to a courseassociated with the course content; determining a correlation betweenthe event count and a level of achievement with respect to the coursebased at least in art on the one or more groups of users; and providingan indication of the correlation between the event count and the levelof achievement with respect to the course.
 5. The method of claim 4,wherein: the representation parameters comprise a time interval, thetime interval being shorter than the period of interest; and determiningthe event count for the property during the period of interestcomprises: separating the period of interest into a set of sub-periodsaccording to the time interval, a length of at least one sub-periodcorresponds to the time interval; and determining a plurality of countsfor the period of interest, each count being the number of eventscorresponding to the property for a sub-period and the event count beinga sum of the plurality of counts.
 6. The method of claim 5, furthercomprising generating a usage indicator for the property based at leaston the event count, the usage indicator being reflective of at least ausage amount of the property during the period of interest, and whereingenerating the usage indicator for the property based at least on theevent count comprises: determining a top count from the two or morecounts, the top count being the count from the two or more counts with agreatest value; and normalizing each count with the top count.
 7. Themethod of claim 6 further comprises: receiving a transformation modeindicator for the event count, the transformation mode indicatorproviding a type of adjustment to be applied to the event count; andvarying each count in accordance with the transformation mode indicator.8. The method of claim 7, wherein: the transformation mode indicatorcorresponds to a peak mode; and varying each count in accordance withthe transformation mode indicator comprises: identifying the sub-periodassociated with the top count; and for each sub-period subsequent to theidentified sub-period, replacing the corresponding normalized count withthe normalized top count.
 9. The method of claim 7, wherein: thetransformation mode indicator corresponds to a cumulative average mode;and varying each count in accordance with the transformation modeindicator comprises, for each sub-period, retrieving the countassociated with each prior sub-period; and generating an adjusted countby determining an average of the count for each prior sub-period and thecount for that sub-period; and replacing the count for that sub-periodwith the adjusted count.
 10. The method of claim 4, wherein determiningthe event count for the property during the period of interest furthercomprises: setting the event count to zero; retrieving, from the one ormore storage components, events that satisfy the representationparameters; and for each retrieved event, determining whether that eventis associated with the property and if so, incrementing the event count.11. The method of claim 4, further comprising generating a usageindicator for the property based at least on the event count, the usageindicator being reflective of at least a usage amount of the propertyduring the period of interest, and wherein generating the usageindicator for the property based at least on the event count comprises:receiving a weight factor for the activity, the weight factor indicatingan amount of influence the activity has on the usage of the electroniclearning system; and adjusting the event count with the weight factor.12. The method of claim 4, wherein determining the event count for theproperty during the period of interest further comprises: receiving acount parameter for the event count, the count parameter providing atleast one limitation to a value of the event count; and adjusting theevent count based on the count parameter.
 13. The method of claim 12,wherein: the count parameter comprises an upper count limit; adjustingthe event count based on the count parameter comprises: determiningwhether the event count exceeds the upper count limit; and in responseto determining the event count exceeds the upper count limit, reducingthe event count to the upper count limit.
 14. The method of claim 12,wherein: the count parameter comprises a lower count limit; adjustingthe event count based on the count parameter comprises: generating anintermediary event count by subtracting the lower count limit from theevent count; and if the intermediary event count is less than zero,setting the event count to zero, otherwise using the intermediary eventcount as the event count.
 15. The method of claim 4, further comprisinggenerating a usage indicator for the property based at least on theevent count, the usage indicator being reflective of at least a usageamount of the properly during the period of interest, and wherein: theactivity is associated with a first course content and a second coursecontent that is different from the first course content; the usageindicator comprises a first usage indicator for the activity at thefirst course content and a second usage indicator for the activity atthe second course content; determining the event count for the propertyduring the period of interest comprises: generating a first event countfor the first course content, the first event count corresponding to theusage amount of the activity at the first course content during theperiod of interest; and generating a second event count for the secondcourse content, the second event count corresponding to the usage amountof the activity at the second course content during the period ofinterest; and generating each of the first usage indicator and thesecond usage indicator based at least on the first event count and thesecond event count.
 16. The method of claim 15 further comprises:determining whether the first event count is greater than the secondevent count; and if the first event count is greater than the secondevent count, normalizing each of the first event count and the secondevent count with the first event count, otherwise normalizing each ofthe first event count and the second event count with the second eventcount.
 17. The method of claim 4, further comprising generating a usageindicator for the property based at least on the event count, the usageindicator being reflective of at least a usage amount of the propertyduring the period of interest, and wherein generating the usageindicator for the property comprises providing a visual representationof the usage indicator.
 18. The method of claim 17, wherein the visualrepresentation comprises a heat map.
 19. The method of claim 17, whereinthe visual representation comprises an adjustable display operable to:receive one or more control inputs for selecting a designated timewithin the period of interest; and varying the visual presentation toillustrate the usage indicator at the designated time.
 20. The method ofclaim 4, wherein the property further comprises a group composed of oneor more users of the electronic learning system.